The present invention relates to methods and systems for measuring the latency between two points connected by a bus with a reference clock synchronous to a processor clock, such as, for example, a PCI Express (“PCIe”) bus, and, more particularly, to methods and systems for accurately measuring latency between two points connected by a PCI Express bus in a combined field programmable gate array (“FPGA”) and conventional processor (“CPU”) processing system.
The advent of FPGA based co-processors for conventional commodity servers has led to a number of issues regarding accurate and high precision processing time measurement.
In systems with computation occurring in both an FPGA as well as a CPU, each with their own independent timing, measuring point to point latencies between both the FPGA and CPU segments is non-trivial due to, for example, clock drift and the difficulty in achieving accurate synchronization during calibration.
As an example of latency concerns, in a system such as financial market data processing, the market data to be processed enters the system via an Ethernet or other connection directly to the FPGA. Once the FPGA has completed its processing, data is then passed to a CPU over a PCIe bus for further processing within or by a trading algorithm.
The amount of time that expires during the FPGA processing is typically measured within the FPGA itself by instantiating a regularly clocked counter. Likewise, in the CPU, Time Stamp Counter (“TSC”) instructions allow checking of a similar regularly clocked counter for measuring times.
However, in measuring time for a combined system, problems occur since the rate at which the FPGA and CPU counters increments can be different. Furthermore, the counters begin at different times. Even if the rate difference and start times can be accounted for in the system, independently running clocks quickly drift out of synchronization, among other concerns. This raises challenges when timing the complete processing from the input of Ethernet data into the FPGA, across the PCIe bus, into a trading algorithm running in the CPU.
Accordingly, there is a continued need for systems and methods that accurately measure latency between two points connected by a PCIe bus in a combined FPGA and CPU processing system.